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MONITORING AND EVALUATING USAGE 
AND COURSE SUMMARY 


INTRODUCTION 


Monitoring usage of a health service is one way that you can assess 
the effectiveness of the service. Monitoring usage of a service means 
regularly checking the extent to which the service is actually being 
used. The information obtained will make it possible to see how much 
services are used, determine whether problems are affecting usage, and 
find ways to increase use. 


Evaluating usage of a health service means periodically (for example, 
once a year) comparing actual usage of the service to the usage 
targets to see if the usage targets were achieved. The information 
obtained will allow you to see if adequate progress was made in 
increasing usage and to increase the accuracy of your future usage 
targets; you will also be able to use the information when ordering 
supplies for the next year. 


Both monitoring and evaluating usage should involve more than simply 
stating that use of a service is increasing or decreasing, or that a 
usage target was achieved. They should also involve finding out why 
usage is up or down or why usage targets were or were not met, and 
taking appropriate action. 


Monitoring and evaluation should both be done at regular times. For 
example, at the end of each month, usage of services could be 
monitored for that month; at the end of each year, usage of health 
services could be evaluated for that year. The best time to evaluate 
usage of health services is just before setting usage targets for the 


coming year. 


LEARNING OBJECTIVE 


The tasks involved in monitoring and evaluating usage are shown on the 
following chart. The information, examples, and practice exercises in 
this module should prepare you to do these tasks in your own health 
area. 


Monitoring Usage 


Be sure the Summarize Analyze the 
data you will the usage | usage data 
need to monitor data each each month. 


usage of health month. 3.0 | 


services is 2.0 
collected. 


Evaluating Usage 


At the end of Compare usage rates 
the year, to targets and 
calculate usage determine reasons 


rates for health for differences. 
services. 6.0 


5.0 


HOW THIS MODULEIS ORGANIZED 


This module is presented in two parts: 


PART I: 


PART ITI: 


There is 
presents 


Take action 
in response 
to monthly 

monitoring. 


4.0 


Take action 
in response 
to annual 

evaluation. 


7.0 


MONITORING USAGE OF HEALTH SERVICES (Steps 1.0 - 4.0) 
describes how to monitor usage of services each month. 


EVALUATING USAGE OF HEALTH SERVICES (Steps 5.0 - 7.0) 
describes how to do an annual usage evaluation. 


also a COURSE SUMMARY at the back of the module which 
the main points of each module of this course. 


PART I: MONITORING USAGE OF HEALTH SERVICES 


1.0 BE SURE THE DATA YOU WILL NEED TO MONITOR USAGE 


OF HEALTH SERVICES IS COLLECTED. 


1.1 DETERMINE WHAT DATA YOU WILL NEED. 


1.2 


To monitor usage of a health service, you will find out how many 
times the service is actually used each month. To be able to do 
this, you will need certain data on the patients seen at your 
health facility and seen by the community health workers in your 
health area. The data which should be collected on each patient 
include: 


@e the name of the patient, 


@e the date of visit (so you will know if usage was in the month 
you are monitoring), 


@ the age or birthdate of the patient (so you will know which 
patients are in the target population of the service), and 


@® the service provided to the patient (so you will know which 
service was used). 


CHECK EXISTING FORMS (FOR EXAMPLE, PATIENT REGISTERS) TO DETERMINE IF 
THE DATA NEEDED IS ALREADY BEING COLLECTED. IF NECESSARY, MODIFY EX- 
ISTING FORMS TO OBTAIN THE NECESSARY DATA. 


If you modify forms already used at the facility or used by a 
community health worker, be sure the health workers who will fill 
out the forms know how to obtain and record the additional 
information on each patient. To do this, provide each worker 
with examples of one or two completed forms as modified. Help 
them practice filling out one or two forms, and carefully observe 
them as they fill out another form to be sure they know how to do 
jt. Observe the health workers closely the first few weeks they 
use the modified forms to be sure they are filling them out 
correctly. 


Emphasize that, if possible, complete information for each visit 
should be recorded during the visit or immediately after a 
patient has been seen. 


2.0 SUMMARIZE THE USAGE DATA EACH MONTH. 
2.1 DEVELOP OR OBTAIN MONTHLY USAGE FORMS. 


Monthly usage forms for preventive and treatment services will be 
your record of the actual usage of services offered by the health 
facility and community health workers in your health area. 
(Examples of monthly usage forms are provided on the next 2 
pages.) The forms should list the health services offered in 
your area, and the target populations for those services. There 
should be a place on the forms to record the number of uses of 
each health service during the month (total and by the target 
population), for the health facility and the community health 
workers and for the health area as a whole. 


MONTHLY USAGE FORM 


PREVENTIVE SERVICES 


HEALTH AREA 


MONTH YEAR 


NUMBER OF USES OF PREVENTIVE SERVICES DURING THE MONTH 


IMMUNIZATION em, 
Measles DPT-3 Other 


em ii 4 Target 
ei Population 


eh bc Target 
echt Population 


Target Erotetl os Target cae ki Target hers  j. Target 
ae: Population eateks Population bese. Population 


Health 
Facility 


Community 
Health Worker 


Community 
Health Worker 


*Target population definitions may vary. 
**You can include separate columns for each family planning service offered in your health area. 


TOTAL IN 
HEALTH AREA 


MONTHLY USAGE FORM 


TREATMENT SERVICES 


HEALTH AREA 


MONTH YEAR 


NUMBER OF USES OF TREATMENT SERVICES DURING THE MONTH 


DISEASES PREVENTABLE BY IMMUNIZATION DIARRHOEAL DISEASES 
MALARIA TRAUMA 
New Antibiotics TREATMENT 
ae Episodes OR IV or Other 
Measles Pertussis Other Treated Therap Therap Drugs 
By By By By By By By By 
Target soca {THEE Target Target Target Target Target tara | 
Pop.*| Total Pop. Total Pop. Total Pop. Total Pop. Total Pop. Total Pop. Total Pop. Total 


RESPIRATORY 
ILLNESS 


UNDER- 
NUTRITION 


Health 
Facility 


Community 
Health Worker 
in Sansui 

Village 


Community 
Health Worker 
in Moulaba 


Village im | ie | iT 
TOTAL IN 
HEALTH AREA 


*Target population definitions may vary. 


2.2 


COMPLETE THE MONTHLY USAGE FORMS. 


To 


complete a monthly usage form for a month: 


Label the form with the name of the health area and the month 
and year you are monitoring. 


Review, for the month, the patient register kept in the 
health facility. 


Count in the patient register the total number of uses of 
each health service at the facility during the month. Then, 
in the row, "Health Facility", enter the total number of uses 
of each service at the facility. 


(Note that a use may be defined differently for different 
services. For example, with diarrhoea treatment service you 
may wish to count each new episode of diarrhoea treated as 
one use. For other services you may wish to count one visit 
as one use. Be sure to define "use" here the same way you 
did when setting targets.) 


Count in the patient register the number of uses of each 
health service by the specified target populations during the 
month. Then, in the row, "Health Facility", enter the number 
of uses of each health service by the target population at 
the facility. 


Review, for the month, the patient register kept by each 
community health worker. 


Repeat steps C and D above for each community health worker 
in the health area. Enter the usage figures in the 
appropriate rows on the form. 


For each health service, add the total number of uses of the 
service in the health area. Enter the sum in the row, "TOTAL 
IN HEALTH AREA." 


For each health service, add the number of uses by the target 
population in the health area. Enter the sum in the row, 
"TOTAL IN HEALTH AREA." 


EXAMPLE The supervisor of Bornu Health Area was completing 
his monthly usage form for preventive services for March 1984 for 
the measles immunization service. To do this, he did the 


following: 


1. 


First, he reviewed the patient register kept at the health 
centre, and counted the number of measles immunizations given 
at the centre between March 1 and March 31. (Look at the 
entries in the patient register for March 1 and 2 on pages 12 
and 13. Notice how the supervisor circled each use of the 


measles immunization service.) 


He entered the total number of uses of the service at the 
centre for the whole month, that is, 35, on his monthly usage 
form in the row, "Health Centre." (See his monthly usage 


form on page 14.) 


Next, he looked in the patient register at the age of each 
patient who was given a measles immunization, and identified 
all children who were in the target population for the 
service, that is, children 9 to 12 months of age. 


He entered the number of uses of the measles immunization 
service by children 9 to 12 months of age for the month, that 
is, 30, in the row, "Health Centre." (See the monthly usage 
form. ) 


To determine the number of uses of each service for each 
community health worker, he reviewed the patient register 
kept by each health worker and repeated steps 1, 2, 3, and 4 
above. He entered these usage figures in the appropriate 
rows on his form. 


To determine the total number of uses of the measles 
immunization service in the entire health area in March, he 
added the number of uses in the health centre (35) to the 
numbers of uses for the community health worker in Sansui 
Village (9), and the community health worker in Moulaba 
Village (7). He entered this sum (51) in the row, "TOTAL IN 
HEALTH AREA." 


To determine the total number of uses of the service by 


children 9 to 12 months of age in the entire health area, he 


added the number of uses by the target population in the 
health centre (30) to the number of uses for the community 
health worker in Sansui Village (7) and the community health 
worker in Moulaba Village (6). He entered this sum (43) in 
the row, "TOTAL IN HEALTH AREA." 


EXERCISE A 


In this exercise you will obtain usage data from a patient register 
and will complete a monthly usage form for the diarrhoea treatment 
service in Bornu Health Area. 


Note: 


In Bornu Health Area, each new episode of diarrhoea treated is 

counted as one use of the diarrhoea treatment service. As you 

do Step 1 of this exercise, remember not to count repeat visits 
for the same episode of diarrhoea. 


1. Refer to the patient register for March 1 and 2 for the health 
centre on pages 12 and 13, and answer the following questions: 


Ae 


How many new episodes of diarrhoea were treated at the health 
centre on March 1 and 2? How many of these were in 
the target population for the diarrhoea treatment service 
(that is, in children less than 5 years)? 


How many of the new episodes of diarrhoea were treated with OR 
therapy on March 1 and 2? How many of these were in 
children less than 5 years? 


How many of the new episodes of diarrhoea were treated with IV 
therapy on March 1 and 2? How many of these were in 
children less than 5 years? 


How many of the new episodes of diarrhoea were treated with 
antibiotics or other drugs on March 1 and 2? How many 
of these were in children less than 5 years? 


2. To determine actual usage of the diarrhoea treatment service in 
the centre for the entire month of March, the supervisor of Bornu 
Health Area reviewed all entries made in the register from March 1 
to 31. He found that during March, 


a total of 102 new episodes of diarrhoea were treated at the 
health centre, and 77 of these were in children less than 5 


years. 


95 new episodes of diarrhoea were treated with OR therapy 
(either alone or with other treatment) at the centre, and 77 of 
these were in children less than 5 years of age, 
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e@ 10 new episodes of diarrhoea were treated with IV therapy 
(either alone or with other treatment), and 3 of these were in 


children less than 5 years, and 


@ 2 new episodes of diarrhoea were treated with antibiotics or 
other drugs (either alone or with other treatment), and both of 
these were in children less than 5 years. 


Fold out the monthly usage form for treatment services on 
page 15. Enter the above usage figures in the appropriate row on 
the form. 


Note: The number recorded in a column for a type of treatment 
would reflect the number of new episodes which received that 
specific treatment, with or without other treatment. For example, 
a new episode of diarrhoea treated with OR therapy alone would 
only be recorded in the column "OR Therapy." But a new episode of 
diarrhoea treated with both IV and OR therapy would be recorded in 
both of those columns. In either situation, each new episode of 
diarrhoea would be counted only once for the column "New Episodes 
Treated." As a result, when added together, the numbers recorded 
on the form in the columns "OR Therapy," "IV Therapy," and 
"Antibiotics or Other Drugs" might not total the number in the 
column "New Episodes Treated." 


To determine actual usage of the diarrhoea treatment service for 
the community health workers in his area during March, the 
supervisor reviewed the patient register kept by each health 
worker. He found that from March 1 to Si, 


@ 51 new episodes of diarrhoea were treated by the community 
health worker in Sansui Village, and 41 of these were in 
children less than 5 years. All were treated with OR therapy. 


@ 47 new episodes of diarrhoea were treated by the community 
health worker in Moulaba Village, and 38 of these were in 
Children less than 5 years. All were treated with OR therapy. 


Enter the above usage figures for the diarrhoea treatment service 
for the community health workers in Bornu Health Area on the 
monthly usage form on page 15. 


ae On the form, calculate and record the total number of new 
episodes of diarrhoea which were treated in Bornu Health Area 
between March 1 to 31. Calculate and record the number of new 


episodes of diarrhoea treated in children under 5 in the area 
in March. 


b. Calculate and record the total number of new episodes of 
diarrhoea which were treated with OR therapy in Bornu Health 
Area between March 1 to 31. Calculate ana record the number 


of new episodes of diarrhoea in children under 5 treated with 
OR therapy in the area. 
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ce. Calculate and record 
diarrhoea which were 
Area between March 1 


the total number of new episodes of 
treated with IV therapy in Bornu Health 
to 31. Calculate and record the number 


of new episodes in children under 5 treated with IV therapy in 


the area. 


d. Calculate and record 
diarrhoea which were 
in Bornu Health Area 
record the number of 
under 5 treated with 


the total number of new episodes of 
treated with antibiotics or other drugs 
between March 1 to 31. Calculate and 
new episodes of diarrhoea in children 
antibiotics in the area. 


Be sure these sums are entered in the row "TOTAL IN HEALTH AREA" 
on the monthly usage form. 


If you have brought a form which is used in your area to record 
the number of times a service is used each month, get it out now. 
Compare it to the Monthly Usage Forms in this module. Discuss it 
with the course facilitator when you discuss this exercise. 


When you have finished this exercise, talk with the 


course facilitator. 
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Health Centre Patient Register 


Bornu Health Area 
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Patient Register 


Month Marly 
Year 934 


Visit Name of Patient Patient Reason for Visit Visit? Services Provided 
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MONTHLY USAGE FORM 


PREVENTIVE SERVICES 


NUMBER OF USES OF PREVENTIVE SERVICES DURING THE MONTH 
IMMUNIZATION 


cate saat | 
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Health 
Centre 


Community 
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Community 
Health Worker 
in Moulaba 
Village 


Hele tf bl fs 
rll ble [a 


TOTAL IN 
HEALTH AREA 


*Target Population = All Ages 
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MONTHLY USAGE FORM 


TREATMENT SERVICES 


NUMBER OF USES OF TREATMENT SERVICES DURING THE MONTH 


BY IMMUNIZATION DIARRHOEAL DISEASES 
MALARTA 
New Antibiotics TREATMENT 
Episodes OR IV or Other 
ssis Tetanus Treated Therap Therap Drugs 
Less Less Less Less Less Less Less 
Than Than Than Than Than Than 
Total | 1 Year| Total| 5 Years | Total | 5 Years| Total Total | 5 Years Total| 5 Years 


RESPIRATORY 
ILLNESS 


Than 


a | |. 
ae 


LD 


Bornu 
Health 
Centre 


Community 
Health Worker 
in Sansui 
Village 


Community 
Health Worker 
in Moulaba 
Village 


TOTAL IN 
HEALTH AREA 


*Target Population = All Ages 


2.3 KEEP GRAPHS ON USAGE OF EACH SERVICE. 


There are two types of graphs which you can keep to monitor usage 
of a health service: a monthly usage graph and a cumulative 
usage graph. These graphs should be made at the beginning of 


each year and filled in each month. 
@ Keep a monthly usage graph for health services. 


A monthly usage graph for a service will show the total number 


of uses of that service by the target population by month. 


This type of graph will allow you to see differences in usage 
of the service from month to month. It will be important to 
find out the reason for these differences. After keeping a 
monthly usage graph on a service for a few years, you will be 
able to observe seasonal patterns in usage of the service in 
your area. Knowledge of seasonal patterns in usage will be 
helpful in setting targets and ordering supplies. 


EXAMPLE 


Monthly Usage Graph for 
Measles Immunization Service in Bornu Health Area, 1984 


80 

? eee ee 

60 
Total Number of 509 | lee | | [dT Tt Ee 
Immunizations 40 a oe etc. ee 
Given to 30 PES : 
Chalaran Sith sions aes 


She 
Sto? 3 


month 7 —eaeee 
OnE: S 0 EES rary 
ALS 


The letters on the bottom line of the graph represent the 
months of the year. The numbers on the vertical line at the 
left (that is, the numbers Oy 10,20 . . eee represent total 
number of uses of the health service by the target 

population. These numbers may be different for each health 
service you monitor. It is important to remember that the 
number at the very top of this line should always be greater 
than the number of expected uses of the service for any month, 


Notice that the graph in this example has been filled in for 
three months. The graph shows that 34 measles immunizations 
were given to children 9 to 12 months of age in Bornu Health 
Area in January, 52 in February, and 43 in March. 
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To fill in a monthly usage graph for a month, 


(1) obtain from the monthly usage form the total number of 


uses Of the service by the target population during the 
month ("TOTAL IN HEALTH AREA") 


(2) locate the number of uses on the vertical axis, and 
(3) above the letter for the month, draw a bar to that number. 
@ Keep a cumulative usage graph for health services. 


It may be helpful to keep a cumulative usage graph for those 
health services where there is not a seasonal variation in 
need or demand for the service (for example, immunization 
services). 


A cumulative usage graph for a service will show the 


cumulative number of uses of the service through the year. 
This means that the point for a month on a cumulative graph 
represents the sum of the number of uses of the service for 
that month plus the number of uses for all previous months of 
the year. With this type of graph, you will be able to check 
progress through the year toward achievement of your 
end-of-year usage target. 


A cumulative usage graph may not be helpful for monitoring 
services where there is a known seasonal variation in 
occurrence of the disease and thus in need or demand for the 
service, such as diarrhoea treatment. It will be difficult to 
see if you are making adequate progress toward achieving a 
target for the service because progress toward the target may 
not occur at a regular rate. With experience, however, you 
will be able to estimate the seasonal variations and consider 
them in determining whether cumulative usage is "on target." 


EXAMPLE On the following page is an example of a cumulative 
usage graph for the measles immunization service in Bornu Health 


Area. 
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Notice that a major difference in the appearance of this graph 
and the monthly usage graph on page 16 is that this graph has 

a line drawn across it. When you first make a cumulative 

usage graph, you should plot a point on the graph on the 
vertical line above "D" for December, directly across from the 
expected usage for the service for the year (in this example, 
that number is 550). Then draw a straight line from the zero 
(0) to the point. This line could be called the "target line." 


Cumulative Usage Graph for 
Measles Immunization Service in Bornu Health Area, 1984 
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Notice also that this cumulative usage graph has been filled 
in for 3 months. According to the graph, cumulative use of 

the measles immunization service in Bornu Health Area was as 
follows: 


| _Monthly Usage _| Cumulative Usage 
ee ae See 
ee: GB: .. . fer 43) 


To fill in a cumulative usage graph for a month, 


(1) Add the total number of uses of the service by the target 
population during the current month to the total number of 


uses of the service by the target population during all 
Previous months of the year. 


(2) Locate the number of uses on the vertical axis, and plot a 


point across from that number on the vertical line above 
the month. 


Ans 


EXERCISE B 


In this exercise you will complete a monthly usage graph for one month 
for the diarrhoea treatment service in Bornu Health Area. 


1. 


Total Number 
of Episodes 
Treated in 
Children 
under Age 5 


Look at the monthly usage form for treatment services on 

page 15. What was the total number of uses of the diarrhoea 
treatment service by children less than 5 years of age in Bornu 
Health Area in March? (Remember: In Bornu Health Area, this 
figure will be the total number of new episodes of diarrhoea 
treated in children less than 5 years.) Write your answers in the 
space below. 


Below is a monthly usage graph for the diarrhoea treatment service 
for Bornu Health Area. Bars showing usage of the service for 
January and February have already been drawn. Draw a bar on the 
graph to show usage for March. 


Monthly Usage Graph for 
Diarrhoea Treatment Service in Bornu Health Area, 1984 


When you have finished this exercise, talk with the 


course facilitator. 
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3.0 ANALYZE THE USAGE DATA EACH MONTH. 


You will plot usage on your usage graphs each month. After you 
have been keeping the graphs for 3 or 4 months, you should start 
looking for monthly differences in usage and start assessing 
progress toward achievement of your targets. 


3.1 DETERMINE IF THERE HAS BEEN A BIG INCREASE OR DECREASE IN 
USAGE OF A SERVICE FROM ONE MONTH TO THE NEXT. 


Look at your monthly usage graph. Compare usage during the 
current month to usage for previous months of the same year. 
After you have kept graphs for a year or more, you can also 
compare actual usage for the current month to usage for the same 
month in previous years. 


If you notice that there has been a big change in usage of a 
service for a month and you know it is due to a seasonal pattern, 
then you will not need to investigate to find the reason for the 
change. 


3.2 DETERMINE IF SATISFACTORY PROGRESS IS BEING MADE TOWARD ACHIEVE! 
OF THE USAGE TARGET FOR THE SERVICE. 


If you are keeping a cumulative usage graph for the service, 
notice where the points are that you plotted for each month. Are 
they all close to the target line? This would indicate that 
satisfactory progress is being made. Is the point for the 
Current month much lower than the target line? This would 
indicate that satisfactory progress is not being made. 


3.3. IF THERE HAS BEEN A BIG INCREASE OR DECREASE IN USAGE FOR THE 
MONTH 
IF IT APPEARS THAT SATISFACTORY PROGRESS IS NOT BEING MADE TOW 
ACHIEVEMENT OF YOUR USAGE TARGET, INVESTIGATE TO FIND OUT WHY. 


You can investigate in the following ways: 


@ talking with health workers, with your supervisor, or with 
Supervisors of other health areas, and looking at usage data 
from previous years to see if there is some good explanation 
(for example, a known epidemic or seasonal trend), 


@ checking for counting errors that might have been made when 


be ee number of uses of the service for the month (step 
e ’ 
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REASONS 


REASONS 


reviewing records of monthly monitoring of performance of 


health workers (see Monitorin Performance) to see if any 


problems or achievements you identified might be affecting 
usage of the service (for example, you may have found that a 
new health worker was diagnosing patients' illnesses 
incorrectly or was incorrectly recording service provided), 


discussing clinic practices related to the service with health 
workers, 


asking health workers about who has been coming for the 
service and who has not, 


talking with mothers and community decision makers to see how 
the community is reacting to the service. 


USAGE MIGHT BE LOW: 


there may be a seasonal low in need for service (episodes of 
diarrhoea) or in use of service (transportation is difficult 
during rainy season) ; 


Overall demand for service may have decreased (for example, 
the number of episodes decreased so there were not as many 
episodes to be treated), 


clinic practices may be discouraging or limiting use, 
adequate supplies may not have been available, 


people may not be using services for social or economic 
reasons, 


records may not be correct, 


diagnoses may have been wrong (for example, patients who 
actually had the disease may have been diagnosed as having 


another disease), 


plans for increasing access or usage may not have been carried 
out (for example, a community heaith worker may not have been 
assigned to a village as expected), 


promotional activities may not be appropriate or may not be 
reaching the right group of people. 


USAGE MIGHT BE HIGH: 


there may be a seasonal high in disease occurrence or in use 
of service (for example, if there has been an epidemic), 


overall demand for service may have increased (for example, if 
a private physician has left the area), 


@ clinic practices may be encouraging use (for example, 
convenient clinic hours), 


@® target population of service may have increased, 


@ records may not be correct, 


@ diagnoses may have been wrong (for example, patients who did 
not have the disease may have been diagnosed as having it), 


@ promotional activities may be starting to reach the right 


group Of people. 


EXAMPLE When the supervisor of the Bornu Health Area 
compared usage of the DPT immunization service for March 1984 to 


usage of the service 
there was a very big 
what caused this big 
his area involved in 
vaccine stock cards. 


The supervisor found 
ordered in June 1983 


for January and February, he noted that 

increase in usage in March. To find out 
increase, he spoke with health workers in 
immunization activities and reviewed the 


that the supply of DPT vaccine which was 
and should have arrived in January 1984 did 


not arrive until early March. Children brought for immunizations 
to the health facility and to the outreach workers in the 
villages of Sansui and Moulaba in January and February were given 
measles and BCG immunizations as needed. Mothers were told to 


bring their children 


back for DPT immunizations in March. 
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EXERCISE C 


In this exercise you will compare monthly usage of the diarrhoea 
treatment service in another health area, the Mamba Health Area, for 
April 1984 to usage of the service in April 1982 and 1983. If you 
think there is a big difference, you will suggest likely reasons for 
the difference. You will then meet as a group to discuss your answers. 


This is the first year (1984) that OR therapy is being offered in the 
Mamba Health Area. The supervisor of the area has been keeping a 
monthly usage graph for the diarrhoea treatment service for 4 months. 
He wanted to know how usage for April 1984 compared to usage for April 
of previous years, so he took the 1982 and 1983 usage data that he had 
used to write his 1984 usage targets and made graphs showing monthly 
usage of the service during those years. 


The usage graphs for the diarrhoea treatment service in the Mamba 
Health Area for 1982, 1983, and January - April 1984 are shown below. 
Look at these graphs, and compare actual usage of the service for 
April 1984 to usage in April 1982 and 1983. Then answer the questions 
on the next page. 


Monthly Usage Graphs for 
Diarrhoea Treatment Service in Mamba Health Area 


Total Number 
of Episodes 
Treated in 
Children 
Under Age 5 


Months 


Total Number 
of Episodes 
Treated in 
Children 
Under Age 5 


Months 
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1. Do you think usage of the diarrhoea treatment service in the 
Mamba Health Area for April 1984 is much more or about the 
same as usage for April 1982 and 1983? Explain. 


2. If you think usage is much more for April 1984, think of at 
least 4 possible reasons for the difference. Write the 
reasons below. 


Tell the course facilitator when you are ready for 


& group discussion of this exercise. 
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4.0 TAKE ACTION IN RESPONSE TO MONTHLY MONITORING. 


4.1 USEINFORMATION FROM YOUR MONTHLY ANALYSIS TO RESOLVE ANY PROBLEMS 
AFFECTING USAGE OF SERVICE. 


If there has been a big increase or decrease in usage of a 

service, appropriate action Should be taken (unless the only 

reason for a difference is a known seasonal high or low in need 

for the service). Action should also be taken if it appears 

that you are not making Satisfactory progress toward achievement of 
your year-end usage target for a service. 


Here are some examples of actions which might be necessary. 


e If you found that people can Only come to the health facility 
at certain times or on specific days, you might adapt the 
hours that the clinic is open to suit the community. 


@ If you found that women are not uSing a service because there 
are only male workers and in their culture there are taboos, 
hire female health workers. 


e If you found that the health facility cannot be reached by 
public transport, you might consider asking the local bus to 
include the health facility on its route. 


e@ If you found that certain health workers are not liked by the 
community, you might need to remove the workers from their 
jobs. Consider district and tribal conflicts before replacing 
the workers. 


@e If you found that overall demand for the service is much more 
than expected, you might need to order more supplies so you 
can serve the increased number of people. 


@e If you found that health workers have been diagnosing patients 
incorrectly (resulting in records that show increased usage 
of one service and decreased usage of another), then you will 
need to take action as described in Monitoring Performance to 


solve the problem. 


4.2 PROVIDE FEEDBACK TO HEALTH WORKERS. 


In addition to resolving any problems you might have found which 
affected usage, you should provide feedback to health workers. 
Feedback can include informing health workers about the 
effectiveness of a health service, providing them with 
information that might be helpful in solving problems, and 
congratulating them on doing a good job or encouraging them to do 


a better job. 


We are making good 
progress toward our 


goals 


4.3. IF REQUESTED, REPORT RESULTS OF MONTHLY MONITORING OF USAGE TO YO 
DISTRICT SUPERVISOR. 


You may be asked to report results of monthly monitoring of usage 
to your district supervisor. If so, you might include the 
following information in your report: 


@ monthly usage totals for each service, 


reasons identified for significant increases or decreases in 
use, if any, 


corrective action to be taken or currently being taken to 
resolve any problems identified, 


action required at a higher level, and 
successful practices or procedures that you would 


recommend for facilities or other community health workers 
in the district. 
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EXERCISE D 


In this exercise you will identify possible actions to take in 


response to monthly monitoring of the diarrhoea treatment service in 
Bornu Health Area. 


In monitoring usage of the diarrhoea treatment service for 

April 1984, the supervisor of Bornu Health Area determined that there 
was a big increase from previous months usage. The possible reasons 
for the increase that he identified are listed below. Write at least 
1 action the supervisor might take in response to each situation. 


Possible Reasons 
for Increase Possible Actions 


OR therapy is being offered 
for the first time in 1984 in 
the area. It has proven 

to be very popular, and 

more mothers are bringing 

in their children for 
treatment than expected. 


Health workers who have 
just been trained to 
provide OR therapy are 
very enthusiastic about it. 


The promotional campaign 
for OR therapy has been 
effective. 


When you have finished this exercise, talk with 
the course facilitator. 
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PART II: EVALUATING USAGE OF HEALTH SERVICES 


V\\\, 


NOTE: Steps 5.0 - 7.0 would be done at the end of 
the year only. 


5.0 AT THE END OF EACH YEAR, CALCULATE USAGE RATES F 
HEALTH SERVICES. 


The following data are needed to calculate usage rates for a 
service: 


@ total number of times the service was used during the year 
being evaluated, 


@ desired usage of the service by the target population with 
access (calculated in Targets), and 


@ desired usage of the service by the target population in the 
entire health area (calculated in Targets). 


5.1 CALCULATE THE USAGE RATE FOR THE TARGET POPULATION WITH 
ACCESS TO THE SERVICE. 


Divide the actual usage Of a service (that is, the total number 
of times the service was used) by the desired usage of that 
Service by the target population with access. Express the result 
aS a percentage. (Desired usage Of a Service by the target 
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population with access was det 


ermined at fnnd 
year, and should be recorded at the beginning of the 


on your "Targets and Achievements" 


actual usage a usage rate for 


desired usage by target the target population 
population with access with access 


EXAMPLE The supervisor of Bornu Health Area needed to 
calculate the usage rate for children under age 1 with access to 
the measles immunization service in 1984. He knew that 506 
measles immunizations were given to the target population from 
January through December 1984. He obtained his earlier estimate 
of desired usage by the target population with access, which was 
920, from his "Targets and Achievements" form for the year 
January 1984 through December 1984. (Fold out page 37. Notice 
that this number has been circled on the form.) Using this 
information, the supervisor calculated the usage rate as follows: 


e 
506 ag 920 = os 50 = 55% 
(actual usage) (desired usage (usage rate for 
by target children under 
population age 1 with 
with access) access) 


After calculating this actual usage rate, he recorded it on the 
section of the form "Complete at End of Year" in the column 
titled "Usage Rate for Those With Access." 


5.2 CALCULATE THE USAGE RATE FOR THE TARGET POPULATION IN THE 
ENTIRE HEALTH AREA. 


Divide actual usage of the service by desired usage for the 
entire health area. 


actual usage = usage rate 
desired usage for for the 
entire health area health area 


EXAMPLE The supervisor of Bornu Health Area knew that 506 
measles immunizations were given to the target population in his 
area in 1984. Earlier in the year he had estimated that desired 
usage of the service for the entire health area was 1,000. 
(Notice that this number has also been circled on the “Targets | 
and Achievements" form on page 37.) Using this information, he 
calculated the usage rate as follows: 


506 — 1,000 = 0.51 = 518 
(actual usage) (desired usage (usage rate 
for health area) for health area) 


After calculating this usage rate for his entire health area for 
1984, he recorded it on the form in the column titled "Usage Rate 
’ 


for Health Area." 
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EXERCISE E 


In this exercise you will calculate the actual usage rates for the 
diarrhoea treatment service in Bornu Health Area in 1984. 


l. 


Read the following information about Bornu Health Area: 


@ 2,699 episodes of diarrhoea in children under age 5 were 
treated in the Bornu Health Area in 1984. 


Complete the following worksheet. When you need data, refer to 
the information above and to the "Targets and Achievements" form 
On page 37. 


NOTE: Express your answers on the worksheet as whole numbers or 
decimal fractions, as you did in Targets. Express the decimal 
fractions so they have 2 digits to the right of the decimal point 
(for example, 0.18 instead of 0.184). 


As you complete this exercise, remember that you SET targets for the 
diarrhoea treatment service at the BEGINNING of 1984, and are 
EVALUATING these targets at the END of 1984. 
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EVALUATION WORKSHEET 


falta: ain RATE FOR THE TARGET POPULATION WITH ACCESS TO THE 
VICE (in this case, diarrhoea treatment). 


A-l How many times was the service actually 
used in the past 12 months? 


actual usage 


A-2 Record actual usage of the service 
on the "Targets and Achievements" form 


on page 37 in the column titled "Actual 
Usage." 


A-2 At the beginning of the year, what was 
estimated to be desired usage for the 
target population with access? 
desired usage for 
target population 
with access 


A-4 Calculate the usage rate for those 
with access by dividing as follows: 


actual usage 


desired usage for 
target population 
with access 


usage rate for 
target population 
with access 


A-5 Express this usage rate as a percentage 
and enter it on the "Targets and 
Achievements" form in the column 
titled "Usage Rate for Those with 
Access." 


CALCULATE USAGE RATE FOR THE ENTIRE HEALTH AREA. 


B-1 At the beginning of the year, what was 
estimated to be desired usage of the 
service by the target population in 
the entire health area? 


desired usage for 
health area 


B-2 Calculate the usage rate for the 
entire health area by dividing as 


follows: 


usage rate for 
the health area 


desired usage for 
entire health area 


actual usage 


B-3 Express the usage rate for the entire 
health area as a percentage, and record 
it on the "Targets and Achievements" 
form in the column titled "Usage 
Rate for Health Area." 


Remember that the usage rates that you calculate at the end of a year will 

be used in setting targets for the coming year. For example, the supervisor 
of Bornu Health Area will consider these usage rates for the diarrhoea 
treatment service for 1984 in setting usage targets for the service for 1985. 


Notice that a copy of the Evaluation Worksheet is included in the Annex at 
the back of this module. This is for you to use when calculating actual 
usage rates for your health area. 


When you have finished this exercise, talk with the 


course facilitator. 
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6.0 COMPARE USAGE RATES TO TARGETS AND DETERMINE 


6.1 


6.2 


REASONS FOR DIFFERENCES. 


COMPARE ACTUAL USAGE OF A SERVICE TO USAGE TARGETS FOR THAT SERVICE. 


For each health service being evaluated, compare the usage rates 
calculated in steps 5.1 and 5.2 to the targets for that service. 


Determine if there is a major difference between the usage rates 
and targets. 


IF THERE IS A MAJOR DIFFERENCE BETWEEN ACTUAL USAGE RATES 
AND TARGETS, IDENTIFY PROBABLE REASONS FOR DIFFERENCES. 


In many cases you will find that by monitoring usage of the 
service during the year, you have already identified the 
circumstances which led to the differences between actual usage 
rates and targets for a service. If this occurs, you will not 
need to investigate further. 


A major reason for differences in rates at the end of a year may 
be that data used in setting the targets were not accurate. The 
usage data that were available when you first set the targets may 
have been incomplete or of questionable accuracy, or there may 
have been no helpful data. As a result, the first few times a 
comparison is made between targets and usage rates, differences 
may be very large. 


Therefore, if the usage was much less than the target, this will 
not necessarily mean that something is wrong or that your efforts 
have not been successful. Or, if the usage was much more than 
the target, this will not necessarily mean that everything is 
fine and that your efforts have been a total success. It is 
important to remember that it may take a few years for targets to 
be more accurate. Knowing the reasons for a difference will help 
you to write a more accurate target next time. 


If usage for the year was much more than expected, consider the 
following: 


Has the need for the service increased, for example, has there 
been an epidemic which might have increased the number of 


cases of a disease? 


e Have many people moved into the health area? 


e@ Was community concern about the problem previously 
underestimated? 


33 


If usage for the year was much less than expected, 


Has the need for the service decreased, for example, has the 
number of cases decreased so that perhaps there were not as 
Many cases to be treated? 


Are people treating diarrhoea at home (for example, with sugar 
and salt solution, commercial products) so that perhaps fewer 
cases are coming to the health centre? 


Was your definition of your access area too broad? 


Were people in the access area receiving services from a 
Private provider? 


Was community concern about the problem previously 
overestimated? 
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EXERCISE F 


In this exercise you will campare actual usage of the diarrhoea 
treatment service in Bornu Health Area for 1984 to the usage targets 
for that service to determine if there are major differences. If you 
think there are major differences, you will suggest likely reasons for 
the differences. When everyone is ready, there will be a group 
discussion. 


1. Look at the "Targets and Achievements" form on page 37, and fill 
in the blanks in the statement below. 


e The usage target for those with access to the diarrhoea 


treatment Service was ___&. 
@e The usage rate for those with access was %. 
e The usage target for the health area was %. 
@ The usage rate for the health area was % 


2. Do there appear to be major differences between the usage rates 
and targets? Explain. 


3. If you think there are major differences, think of at least l 
possible reason for the differences. Write your answer below. 


Tell the course facilitator when you are ready for 


a group discussion of this exercise. 


7.0 TAKE ACTION IN RESPONSE TO ANNUAL EVALUATION. 


7.1 USE INFORMATION FROM YOUR EVALUATION TO CORRECT ANY PROBLEMS 


T2 


z 3 


AFFECTING USAGE OF THE SERVICE THAT WERE NOT CORRECTED EARLIER 
IN THE YEAR (SEE STEP 4.0). 


After you have identified the most likely reason for differences 
in usage rates and targets, you will need to identify ways to 
correct that difference--unless the problem affecting usage was 
identified earlier in the year through monitoring and has already 


been corrected. 


USE INFORMATION FROM YOUR EVALUATION TO INCREASE THE ACCURACY 
NEXT YEAR’S USAGE TARGETS AND TO ESTIMATE HOW MANY SUPPLIES 
ORDER. (THE PROCEDURES FOR SETTING TARGETS AND ORDERING SUPPLIES A 


DESCRIBED IN TARGETS. ) 


IF REQUESTED, REPORT RESULTS OF YOUR EVALUATION TO DISTRICT 
SUPERVISOR. 


If you will report results of monthly monitoring of usage to your 
Supervisor, you might include the following additional 
information in the report that you submit the last month of the 
year: 


@ usage rates calculated for each service for the year, 


@ probable reasons for differences in usage rates and targets, 
if any, and 


@® action to be taken or currently being taken to correct any 
Problems identified (for example, revision of next year's 
usage targets). 


Evaluation results from supervisors of all health areas in the 
district can be used by your supervisor to evaluate usage of 
services in the entire district. The results will also be 
valuable for setting targets for usage of services in the 
district for the next year. In addition, this information will 
be a good way to show your district supervisor what has been 
accomplished in your health area. 


After evaluation, provide FEEDBACK to |. 
health workers and the community. 
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TARGETS AND ACHIEVEMENTS 


Year of Report: From Tenuery [9f4 — throughQecember [945 
ao (month) (year) (month) (year) 


‘ities and Health Workers in Health Area (Indicate Number of Each): 


3 Community Health Workers Dispensary Hospital Other 


COMPLETE AT BEGINNING OF YEAR: | compuete ar Ep oF year: | 
Desired Usage 
tora _ e S- 3Se rer att * fer ee ror ah 
‘Population Population Expected | for Those Population in | Usage for for Health Actual | for Those for Health 
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e, assumptions made in setting targets, reasons why targets were or were not achieved!: 


Aa bridge wll be oil depended befuen” Bornu and Pram Fgh fate 
3 rem oftenad ag er a! be awaabk, ard. death immunigabion? 


all Villoges/ te Di inet a 


) , Date Targets Set: Canary 19 fF 
Seeosition: Made T) 
Date Evaluated: Canuary 1965 


3G, 


Health Area: Kornu 


Types of Health Facil 


nl Health Centre 


Target 
Population 
Service Definition 


Treatment of Children 
Diarrhoea Under 
Age 5 


Measles Children 
Immunization Under 
Age 1 


Treatment of Children 
Malaria Under 


Age 5 
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REMEMBER THIS 
ABOUT MONITORING AND EVALUATING USAGE 


Monitoring usage of health services each month should help you find 


and correct problems affecting usage of the services at an early 
stage. 


Evaluating whether usage targets for the year were achieved should 
tell you how much progress has actually been made in increasing 
usage of health services, and should help you to increase the 
accuracy of future usage targets. 


Monitoring and evaluating usage should also include trying to find 
out why usage is up or down or why usage targets were or were not 


met, and taking appropriate action. 


It is important to investigate reasons for both problems and 
successes. 


After monitoring and evaluating usage, you should always provide 
prompt feedback to health workers and the community. 
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ANNEX 


Blank Copy of Evaluation Worksheet 
for Use in Your Own Health Area 


4] 


CALCULATE USAGE RATE FOR THE TARGET PO 


SERVICE. 


A-1l 


How many times was the service actually 


EVALUATION WORKSHEET 


used in the past 12 months? 


Record actual usage of the service 


on the "Targets and Achievements" form 


in the column titled "Actual Usage." 


At the beginning of the year, what was 


estimated to be desired usage for the 
target population with access? 


Calculate the usage rate for those 
with access by dividing as follows: 


actual usage 


desired usage for 
target population 
with access 


PULATION WITH ACCESS TO THE 


actual usage 


desired usage for 
target population 
with access 


usage rate for 
target population 
with access 


A-5 Express this usage rate as a percentage 
and enter it on the "Targets and 
Achievements" form in the column 
titled "Usage Rate for Those with 
Access." 


CALCULATE USAGE RATE FOR THE ENTIRE HEALTH AREA. 


B-1 At the beginning of the year, what was 
estimated to be desired usage of the 
service by the target population in 
the entire health area? 


desired usage for 
health area 


B-2 Calculate the usage rate for the 
entire health area by dividing as 
follows: 


usage rate for 
the health area 


actual usage desired usage for 


entire health area 


43 


B-3 Express the usage rate for the entire 
health area as a percentage, and record 
it on the "Targets and Achievements" 
form in the column titled "Usage 
Rate for Health Area." 


Remember that the usage rates that you calculate at the end of a year will 
be used in setting targets for the coming year. 
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DEFINITIONS OF TERMS 


Access - the opportunity to obtain Or use a service. 
People who have access to a health service are 
those who live near enough to use it. 


Cumulative - increasing in size by repeated addition. For 
example, at the end of February, cumulative use 
of a service thus far for a year would equal 
January use + February use. At the end of 
March, cumulative use thus far for a year would 
equal January use + February use + March use. 


Data - information organized for analysis or used as a 
basis for a decision. To determine how many 
times a service is actually used, certain data 
must be collected on patients who visit each 
health facility and community health worker in a 
health area. 


Epidemic - occurrence of cases of a disease clearly in 
excess of the number normally expected in a 
community or area. 


Evaluate - to compare what was achieved to what was 
expected to be achieved. Annual evaluation of 
usage of health services means comparing actual 
usage of the services each year to the usage 
targets for those services to see if the usage 
targets were or were not achieved, and the 
reasons. 


Feedback - information provided by others, such as a 
supervisor, on the way a person is doing work. 
Feedback informs a health worker of work he is 
doing well and should continue, as well as ways 
to improve other work. 


Health Area - the geographic area assigned to a supervisor by 
the government. A health area may include a 
health facility, community health workers, or 


both. 


to closely observe or check on a routine basis. 


Monitoring performance of health workers in 
delivering a health service and monitoring usage 
of the service are ways that a supervisor of a 
local area can assess the effectiveness of the 


service. 


Monitor - 


an occurrence of a health problem which happens 


New episode - 
: after any previous occurrence has stopped. 


Percentage - 


(%) 


Promotional 
activities - 


Target - 


Target Population - 


Usage - 


Usage rate - 


Actual Usage - 


Desired usage - 


Use of a Service - 


a part of a whole expressed in hundredths. If 
50% of a population is the percentage of people 
that are female, it means that 50 out of every 
100 people are female. The following examples 
show different ways of expressing the same 
meaning: 


50% = 0.50 = 50/100 
42% = 0.42 = 42/100 
4% = 0.04 = 4/100 


To multiply by a percentage, first write it as a 
decimal fraction, for example, 0.50. 


things said or done to show people that 
something is good or desirable. Promotional 


activities for a health service might include 


telling more people about the service and 
benefits of its use, making them want to use it. 


a goal to work towards, expressed as a number or 
rate. 


the people for whom a service is primarily (or 
solely) intended. For example, the target 
population for diarrhoea treatment is children 
under 5; the target population for treatment of 
trauma is all ages. 


the extent to which a service is used. Usage 
can be stated as the number of times a service 


is used or as a usage rate. 


a proportion or percentage calculated as follows: 


usage ss actual usage 
rate desired usage 


the number of times a service is really used. 
Reviewing records such as patient registers is 
one way to determine actual usage of a service. 


the number of times a service would be used if 
members of the target population came when they 
needed the service. Desired usage is usually 
greater than actual usage, since some target 
population members do not use the service. 


what you count to measure usage, for example, 


new episodes treated or times the service is 
given. 
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COURSE SUMMARY 


Controlling childhood death and disability will require long-term 
efforts on the part of a community. If health workers are to 
contribute to these efforts, they must receive adequate supervision 
and support. 


These training materials were designed to teach you supervisory 
skills, that is, skills in PLANNING, MONITORING, TRAINING, and 
EVALUATING. The skills taught in the course are not only for use in 
supervising the treatment and control of diarrhoea, but can also be 
used in other health services such as immunization programmes, the 
treatment of acute respiratory infections, or malaria. 


"Supervisory Skills" includes an introduction and six modules. The 
main points of each module are listed in this summary. Review these 
points and think about what you have learned and what you plan to do 
with your new skills and knowledge. If you work hard to apply these 
skills, you should be rewarded by improvements in the health services 
being provided to your community. 
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COMMUNITY INVOLVEMENT 


Work with the communities in your health area. Be sure 
the communities join in discussions and help make 
decisions about health matters. 


Find out who has access to health services and who does 
not. 


Find out through discussion with the community what 
health problems are of greatest concern to them. 


Discuss with community decision-makers possible ways of 
improving health services and the health of the 
community. 


Reach agreement with the communities on how their 
resources and those of the Ministry of Health will be 
used to improve health. 


TREATMENT OF DIARRHOEA 


Dehydration is the most important problem that must be 
treated in a patient with diarrhoea. 


Families should be educated about proper treatment of 
diarrhoea in the home. This can often prevent 
dehydration from occurring. 


Health workers should: 


- ask, look and feel for the signs of dehydration 

- select a treatment plan based on the degree of 
dehydration 

- check for signs and symptoms of serious problems 
other than diarrhoea and give treatment where 
appropriate. 


A diarrhoea treatment wall-chart is included in this 
course to help health workers remember how to assess 
and treat diarrhoea. 
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TARGETS 


Targets help people to: 


- know the goals they are working towards 
- see their progress and feel proud of their work 
- estimate what supplies and resources are needed. 


People will usually work harder to achieve targets if: 


- the targets are realistic and 
- they have helped to set the targets. 


To set realistic targets for usage of any health 
service, you first need to estimate past usage of the 
service and consider ways to increase both access and 
usage. 


TRAINING 


Training should always include practice of tasks, in a 
situation as similar to the job as possible. 


Before practising the task, learners should receive any 
necessary information and examples. 


Different methods can be chosen to carry out training. 
Learners must receive feedback on their practice in 
order to know how well they are doing a-task and how 
they can improve. 

Evaluate training by watching learners working. If 


they still cannot carry out a task, find out why. Use 
the results of evaluation to improve training. 
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MONITORING PERFORMANCE 


Monitoring the performance of health workers should 
help to discover any problems in delivery of health 
services and help health workers improve their work. 
It should also find work that is being done well. 


Before monitoring, decide what items are important to 
monitor, then decide how and when to monitor each 


item. Checklists will help you remember what to look 
for. 


A rational, step-by-step approach should be used to 
find solutions to problems identified in monitoring. 
Briefly, the steps are to: 


describe the problem; 

identify possible causes of each problem; 

- find a reasonable solution to each problem; 
—- monitor the solution once it is implemented. 


After monitoring, always provide prompt feedback to 
health workers. 


MONITORING AND EVALUATING USAGE 


Monitoring usage of health services each month helps to 
find and correct problems affecting usage of the 
services at an early stage. 


Evaluating whether usage targets for the year were 
achieved should indicate how much progress has actually 
been made in increasing usage of health services, and 
increase the accuracy of future usage targets. 


Monitoring and evaluating usage should also include 
trying to find out why usage is up or down or why 
usage targets were or were not met, and taking 
appropriate action. 


It is important to investigate reasons for both 
problems and successes. 


After monitoring and evaluating usage, feedback should 
be given to health workers and the community. 


i 
a tal vay 
us 


$- 
t 


s° 


B apse: Pq. 


md oPsey 
Mm toc see 


broske 


209 pais 
63 


